Papers with instruction-based learning
David helps Goliath: Inference-Time Collaboration Between Small Specialized and Large General Diffusion LMs (2024.naacl-long)
Copied to clipboard
| Challenge: | Existing studies of diffusion-based language models have been conducted on a smaller scale. |
| Approach: | They propose to scale an autoregressive diffusion model from 0.4B to 13B parameters and propose techniques to improve its training and inference efficiency. |
| Outcome: | The proposed model is able to combine a large general-purpose diffusion model with smaller, but specialized and contextualized diffusion models at inference time. |